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SUBJECT:  Suggestions  for  Monitoring  of  PSRO  Functions  Delegated  to  Hosptials 


A.  Purpose 

The  purpose  of  this  technical  assistance  document  is  to 
provide  PSROs  with  specific  guidance  and  suggestions  for  moni- 
toring the  performance  of  PSRO  functions  which  are  delegated  to 
hospitals.     The  monitoring  responsibility  has  two  major  aspects: 
the  monitoring  of  review  processes  and  the  monitoring  of  review 
impacts.     The  approach  here  adopted  is  that  monitoring  should 
initially  focus  on  two  review  processes:     that  for  performing 
concurrent  review  and  that  for  Medical  Care  Evaluation  (MCE) 
studies.     In  time,  PSRO  monitoring  will  shift  to  an  evaluation 
of  the  impact  on  quality  of  care  and  utilization  of  services. 

B .  Background 

The  goal  of  the  PSRO  program  is  to  assure  that  the  medical  care 
provided  to  Federal  patients  is  of  high  quality  and  reflects  the 
most  appropriate  and  efficient  utilization  of  institutional  health 
care  services.     The  mechanisms  for  achieving  this  goal  include 
the  performance  of  concurrent  review  and  Medical  Care  Evaluation 
studies. 

Section  1155(e)(1)  of  the  PSRO  statute  provides  that  a  PSRO  may 
delegate  its  functions  to  hospitals  to  the  extent  and  for  such  time 
as  the  hospital  demonstrates  a  capacity  to  conduct  effective  and 
timely  review.     Transmittal  No.  11  dated  November  26,  1974,  describes 
the  procedures  and  policies  for  making  initial  delegation  decisions. 
Within  this  framework,  the  PSRO  is  ultimately  held  accountable  for 
assuring  that  delegated  duties  and  activities  are  performed  in  an 
efficient  and  effective  manner.    Monitoring  hospital  performance 
is  one  way  for  the  PSRO  to  meet  this  responsibility.  PSRO 
monitoring  is  a  fluid  process  that  may  range  from  a  partnership 
that  works  for  continual  upgrading  of  the  PSRO  hospital  review 
system  to  a  decision  by  the  PSRO  that  delegation  should  be  modi- 
fied or  withdrawn. 
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The  Bureau  of  Quality  Assurance  will  review  and  evaluate  the 
degree  to  which  a  PSRO  has  met  its  contractual  responsibilities 
for  assuring  the  effectiveness  of  all  health  care  review  which 
it  is  authorized  to  perform.     The  performance  of  the  PSRO  in 
carrying  out  its  monitoring  responsibilities  will,  thus,  constitute 
an  important  element  in  the  evaluation  of  PSROs.  Assessments 
of  overall  PSRO  performance  will  be  undertaken  as  part  of  the 
decisionmaking  process  for  determining  when  a  project  qualifies 
for  operational  designation. 

C.    Developing  a  Plan  for  Monitoring  PSRO  Functions  Delegated 
to  Hospitals 

The  nature  of  monitoring  by  a  PSEO  may  range  from  ongoing 
day-to-day  interaction  between  PSRO  and  hospital  review  staff 
to  a  structured  review  and  assessment  of  hospital  performance  at 
predetermined  intervals,  e.g.,  quarterly,  annually.  Several 
elements  that  should  be  considered  in  planning  the  structured 
assessment  include: 

1.  Specific  Areas  for  Review:    A  PSRO  should  identify  the  areas 
that  require  assessment,  such  as  performance  of  concurrent 
review  within  program  timeframes  or  completion  of  the  re- 
quired number  of  MCE  studies,  to  permit  the  PSRO  to  determine 
whether  delegation  should  be  continued  and  to  meet  its 
statutory  responsibilities.     The  attachment  suggests  some 
specific  areas  for  monitoring  hospital  performance. 

2.  Methods  for  Monitoring  Hospital  Performance:     There  are 
several  methods  which  a  PSRO  might  utilize  in  monitoring  per- 
formance, including  on-site  hospital  visits,  to  conduct  inter- 
views and  review  medical  records  and  information  maintained 

by  review  personnel  or  the  peer  review  committee  as  well  as 
analysis  of  aggregated  PSRO  data. 

3 .  Assignment  of  Roles  and  Responsibilities  of  PSPX)  Staff: 
Such  arrangements  might  identify:   (a)  roles  and  respons- 
ibilities of  PSRO  staff  in  on-site  monitoring  activities  in 
the  hospital;   (b)  the  involvement  of  PSRO  board  or  committee 
members  in  the  participation  or  review  of  monitoring  findings; 
and  (c)  the  role  of  data  staff  in  assisting  the  program 
operations  unit  or  supervisory  review  coordinator  in  analyzing 
hospital  review  activities. 
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4 .  Frequency  of  Monitoring :  Initially,  a  PSRO  may  find  it  necessary 
to  monitor  all  hospitals  in  its  area  with  equal  frequency. 

In  time,  the  frequency  and  intensity  of  monitoring  may  vary 
depending  on  the  review  caseload  of  the  hospital  as  well  as 
the  perceived  or  real  knowledge  of  PSRO  related  problems  in 
the  hospital. 

5 .  Method  for  Communicating  with  Hospitals:  This  element  involves 
planning  for  how  the  PSRO  will  communicate  monitoring  infor- 
mation to  the  hospital  and  may  range  from  how  the  PSRO  will 
notify  the  hospital  of  a  planned  site  visit  to  communicating 
the  findings  gained  from  review  of  the  hospital's  performance. 
Specifically,  a  PSRO  should  consider:     a)  who  in  the  hospital 
should  be  the  recipient  of  the  information,  e.g.,  hospital 
administrator,  chairman  of  a  peer  review  committee,  etc.; 

b)  how  the  findings  will  be  transmitted,  e.g.,  meetings,  written 
reports,  etc.;  and  c)  what  information  will  be  included,  e.g., 
statement  of  the  findings,  basis  for  reaching  the  conclusion 
or  suggestions  for  resolving  the  PSRO  related  problems. 

D .  Approach 

In  order  to  assist  PSROs  in  identifying  specific  monitoring  areas, 
the  attachment  lists  a  set  of  questions  which  a  PSRO  may  want 
to  ask  concerning  performance  of  concurrent  review  and  MCE  study 
functions  delegated  to  hospitals.     The  questions  for  concurrent 
review  should  allow  the  PSRO  to  determine  whether:  a)  review 
decisions  are  appropriate;  b)  review  procedures  meet  program 
requirements  and  are  performed  in  an  efficient  manner;  and  c)  the 
review  system  is  adequately  managed.    For  MCEs,  a  PSRO  should  be 
concerned  that:  a)  the  MCE  studies  conducted  maximize  the  poten- 
tial for  improving  quality  of  care,  and  b)  the  program  report- 
ing requirements  are  being  met.     These  broad  objectives  which 
focus  on  monitoring  the  processes  for  performing  the  delegated 
functions  represent  the  framework  for  the  attachment.  Specif- 
ically, for  each  of  the  above  stated  broad  objectives,  a  set  of 
sub-objectives  and  related  screening  questions  is  provided. 

The  questions  do  not  include  several  compliance  areas  that  would 
have  been  associated  with  the  initial  delegation  decision. 
Specifically,  this  document  does  not  address:  a)  the  requirement 
that  a  substantial  portion  of  physicians  with  active  hospital  staff 
privileges  must  be  members  of  the  PSRO;  b)  prohibition  of  physicians 
with  a  financial  interest  in  the  hospital  from  being  responsible  for 
review  determinations  on  patients  in  that  hospital;  c)  prohibition 
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of  physicians  from  participating  in  the  review  of  their  own 
cases;  and  d)  hospital  compliance  with  the  physician  certification 
requirements  adopted  by  the  PSRO.    A  PSRO  should  develop  methods 
for  ensuring  continued  compliance  with  these  requirements. 
Furthermore,  the  approach  does  not  provide  for  the  use  of  relevant 
information  which  might  be  available  from  Medicare  intermediaries 
and  carriers  or  Medicaid  State  Agencies. 

E.  Attachment 

The  attachment  is  divided  into  two  sections.     Section  A  addresses 
monitoring  of  concurrent  review  and  Section  B  addresses  monitor- 
ing of  Medical  Care  Evaluation  studies.    A  discussion  of  uses  a 
PSRO  may  make  of  each  section  follows: 

Section  A.    Monitoring  Performance  of  Concurrent  Review 

The  questions  posed  in  this  section  should  be  viewed  as  areas  of 
inquiry  which,  if  answered  unsatisfactorily,  might  suggest  a  need 
for  further  investigation  into  the  operations  of  the  concurrent 
review  program.     Where  appropriate,  comments  are  provided  which 
further  explain  the  nature  of  the  question  and  identify  possible 
follow-up  areas  for  action  which  might  solve  the  problem.  For 
example,  an  unsatisfactory  answer  to  the  question:  "For  those 
patients  known  to  be  Federal  beneficiaries  at  the  time  of  admission, 
are  decisions  to  certify  or  deny  admission/continued  stay  completed 
within  the  program  timeframes?"  might  suggest  the  need  for  addi- 
tional review  personnel,  modifying  review  personnel  schedules, 
or  improvements  in  the  communication  system  between  review  personnel 
and/or  the  attending  physicians. 

The  methods  that  PSROs  would  use  to  determine  answers  to  the  questions 
will  vary.     They  might  include  personal  interviews,  review  of  aggre- 
gated data,  review  coordinator  worksheets,  logs,  and  minutes  of 
the  peer  review  committee,  or  on-site  sampling  of  medical  records. 
PSROs  will  have  to  determine  the  most  appropriate  ways  to  answer 
the  questions. 

In  general,  the  questions  do  not  set  a  standard  or  a  measure  for 
defining  a  satisfactory  or  unsatisfactory  answer.    For  example, 
PSROs  themselves  will  have  to  determine  the  range  of  acceptable 
answers  for  the  question:  "Are  PSRO  approved  norms  generally  used 
in  assigning  length-of-stay  review  dates  for  cases  where  a  clear 
diagnosis  is  known?" 
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Section  B.    Monitoring  Performance  of  Medical  Care 
Evaluation  Studies 

Part  I  of  this  section  contains  questions  which  a  PSRO  would  ask 
in  conducting  an  indepth  review  of  the  specific  MCE  study  in 
order  to  determine  the  appropriateness  of  the  study  methods.  An 
explanation  is  also  provided  to  assist  the  PSRO  in  understanding 
the  purpose  of  the  question  and  how  to  determine  whether  a 
satisfactory  answer  is  obtained.     In  using  this  set  of  questions, 
a  PSRO  essentially  "validates"  the  study  process  used  by  the 
hospital  by  going  through  each  of  the  steps  of  the  study.  The 
questions  range  from  reviewing  how  topics  were  selected  to 
identifying  how  the  MCE  study  was  managed. 

Part  II  of  this  section  focuses  on  monitoring  through  review  of 
the  PMIS  MCE  study  reporting  forms.     The  questions  relate  to  deter- 
mining whether  MCE  program  requirements  are  being  met  including: 

a)  submitting  timely  reporting  forms;  b)  performing  the  required 
number  of  studies  and  c)  accurately  reporting  MCE  study  personnel 
costs. 

Additionally,  a  PSRO  could  review  information  on  the  MCE  Abstract 
and  Re-study  forms  to  identify  general  patterns  in  a  hospital's 
studies.    Although  questions  for  this  type  of  pattern  review  are 
not  provided  in  the  attachment,  several  examples  are  as  follows: 

1)  What  is  the  pattern  in  the  method  used  by  the  hospital  to 
select  MCE  study  topics?    Are  all  the  studies  based  on 
the  same  method,  e.g,  perceived  need? 

2)  What  is  the  pattern  in  the  hospital's  use  of  study  findings 
to  initiate  continuing  medical  or  other  health  care 
practitioner  education  programs? 

Patterns  over  time  that  indicate  potential  problem  areas  might 
provide  the  basis  for  indepth  reviews  of  completed  studies. 

F .  Conclusion 

The  requirement  for  PSPvOs  to  develop  and  implement  methods  for 
monitoring  delegated  functions  is  referred  to  in  numerous  BQA 
documents  including:  a)  the  original  conditional  PSRO  application; 

b)  Volume  II  of  the  PSE.0  Contracts  Management  Manual  and  c)  the 
related  contract  deliverable  of  a  monitoring  plan.  Accordingly, 
many  PSROs  have  already  developed  and  implemented  ways  to  carry 
out  their  monitoring  responsibilities.     For  these  PSROs  the 
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information  in  this  technical  assistance  document  may  be  useful  for 
making  revisions  to  or  comparisons  with  their  existing  monitoring 
approach.     The  information  is  not  meant  to  set  a  standard  for  "good 
monitoring"  but  rather  reflects  an  effort  on  the  part  of  BOA  to 
provide  assistance  to  PSROs  where  applicable.    At  a  minimum,  we 
believe  a  PSRO's  monitoring  system  should  allow  it  to  draw  conclu- 
sions on  whether  the  broad  objectives  previously  identified  in 
this  document  are  being  achieved.     Beyond  this,  the  PSRO  decides 
the  appropriate  methods  and  areas  of  inquiry  in  order  to  reach 
these  conclusions. 

We  welcome  your  comments  and  recommendations  on  this  technical 
assistance  document.     The  document  represents  an  expansion  of  pre- 
vious documents  that  address  the  monitoring  responsibility  of  the 
PSRO,  but  it  focuses  upon  only  two  significant  areas  for  consideration. 
Additional  components  of  the  PSRO  monitoring  functions,  for  example, 
ancillary  services  review,  will  be  discussed  in  future  documents  as 
experience  is  gained  in  applicable  review  areas.    Moreover,  PSRO 
monitoring  of  delegated  hospital  review  costs  will  be  dealt  with  in  a 
separate  issuance.    Finally,  as  experience  with  the  present  PSRO  hospital 
review  system  leads  to  changes,  revisions  in  monitoring  delegated 
functions  will  probably  be  necessary. 


Michael  J.  Goran,  M.D. 


Attachment 


Attachment 

Section  A.  Monitoring  Performance  of  Concurrent  Review  - 
Summary  of  Objectives  and  Sub-objectives 

1.  OBJECTIVE:     TO  DETERMINE  WHETHER  REVIEW  PROCEDURES  MEET 

PROGRAM  REQUIREMENTS 

Sub-obj  ectives : 

a.  To  determine  if  the  hospital  is  using  PSRO  approved  criteria 
for  admission  review. 

b.  To  determine  if  the  hospital  is  applying  PSRO  approved  norms 
appropriately  in  assigning  length-of-stay  checkpoints. 

c.  To  determine  if  the  hospital  assures  that  Title  18,  19  and 
5  patients  are  subject  to  concurrent  admission  review. 

d.  To  determine  if  the  hospital  conducts  admission/continued 
stay  review  in  a  timely  manner. 

e.  To  determine  if  the  hospital  follows  the  program  procedures 
for  denial  of  admission/continued  stay. 

f.  To  determine  if  the  hospital  follows  the  program  procedures 
for  handling  reconsideration  reauests. 

g.  To  determine  if  the  hospital  review  personnel  provide  timely 
information  to  the  discharge  planning  staff. 

2.  OBJECTIVE:     TO  DETERMINE  WHETHER  REVIEW  DECISIONS  ARE  APPROPRIATE 
Sub-obj  ective : 

a.     To  determine  if  appropriate  clinical  judgment  is  applied  in 
making  review  decisions. 

3.  TO  DETERMINE  WHETHER  REVIEW  PROCEDURES  ARE  PERFORMED  IN  AN 
EFFICIENT  MANNER 

Sub-obj  ec  t  ive  s : 


a.     To  determine  if  the  hospital  is  making  appropriate  use  of 
review  personnel. 
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b.  To  determine  the  efficiency  of  the  hospital  in  transmitting 
review  information  to  the  paying  agency. 

c.  To  determine  whether  the  patient  abstracts  are  accurately 
completed  and  efficiently  forwarded  to  the  PSRO  and/or  data 
processing  organization. 

d.  To  determine  if  the  hospital  is  performing  concurrent  review 
within  an  acceptable  cost  range. 

TO  DETERMINE  WHETHER  THE  MANAGEMENT  OF  THE  REVIEW  SYSTEM  IS  ADEQUATE 

Sub-obj  ectives : 

a.  To  determine  whether  the  peer  review  committee  is  carrying  out 
its  responsibilities  for  managing  the  concurrent  review  system. 

b.  To  determine  whether  the  peer  review  committee  utilizes  con- 
current review  information  to  make  recommendations  for  improving 
the  utilization  and  quality  of  health  services. 


Section  A.    Monitoring  Performance  of  Concurrent  Review 


1.     OBJECTIVE:     TO  DETERMINE  WHETHER  REVIEW  PROCEDURES  MEET  PROGRAM 
REQUIREMENTS 

a.  Sub-objective:     To  determine  if  the  hospital  is  using  PSRO 
approved  criteria  for  admission  review. 

1.    Are  screening  criteria  other  than  those  approved  by 
the  PSRO  used  in  admission  review? 

Comment :     Review  Coordinators  utilize  screening  criteria 
approved  by  the  PSRO  in  order  to  objectify  review  and 
identify  those  selected  cases  requiring  peer  review.  An 
affirmative  answer  to  this  question  may  suggest  the  need 
for:     a)  additional  Review  Coordinator  training  or 
b)  revision  of  or  additions  to  the  existing  criteria  set. 

b.  Sub-objective:     To  determine  if  the  hospital  is  applying  PSRO 
approved  norms  appropriately  in  assigning  length-of-stay 
checkpoints . 

1.  Are  PSRO  approved  norms  generally  used  for  the  initial  and 
subsequent  length-of-stay  assignment  where  a  clear 
diagnosis  is  known? 

Comment :     In  the  event  a  significant  number  of  cases  is 
found  to  have  length-of-stay  assignments  that  do  not  appear 
to  relate  to  the  PSRO  approved  norms,  further  investigation 
should  be  conducted  to  determine  if  such  variation  is 
justifiable. 

2.  Are  significantly  more  than  or  less  than  50  percent  of  the 
patients  reviewed  upon  expiration  of  the  initial 
length-of-stay  granted  extensions  of  stay? 

Comment :     This  question  addresses  the  appropriateness  and 
adequacy  of  the  PSRO  approved  norms  for  assigning  initial 
and  continued  stay  review  dates.     The  50  percent  is  a 
suggested  benchmark,  particularly,  since  most  PSROs 
normally  use  the  50th  percentile  of  the  PAS  or  other  regional 
normative  data  as  the  point  in  time  when  the  initial  con- 
tinued stay  review  will  take  olace.     The  fact  that  significantly 
less  than  50  percent  of  the  patients,  for  example,  only 
30  percent,  were  granted  extensions  of  stay  may  suggest  the 
need  to  examine  the  PSRO  approved  norms  in  relation  to  the 
actual  length-of-stay  in  the  particular  hospital  under  review. 
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c.  Sub-objective:     To  determine  if  the  hospital  assures  that 

Title  18,  19  and  5  patients  are  subject  to 
concurrent  admission  review. 

1.     What  is  the  pattern  in  the  number  of  cases  which  were 
reviewed  for  admission  certification  later  than  3  days 
after  admission  and  after  discharge? 

Comment :     Generally,  the  admitting  office  of  the  hospital 
has  established  procedures  for  notifying  the  Review 
Coordinator  of  admissions  that  are  subject  to  concurrent 
review.     However,  there  may  be  occasions  when  information 
about  the  payment  sources  is  not  known  at  the  time  of 
admission  and  either  the  patient  is  reviewed  later  during 
the  hospital  stay  or  retrospectively  after  discharge.  A 
frequent  occurrence  of  either  of  these  situations  may 
suggest  the  need  to  develop  a  procedure  whereby  the  ad- 
mitting office  would  notify  the  Review  Coordinator  of 
patients  who  potentially  appear  to  be  Federal  beneficiaries. 
The  3  day  interval  is  used  to  account  for  admissions 
on  a  non-working  day. 

d.  Sub-objective:     To  determine  if  the  hospital  conducts 

admission/continued  stay  review  in  a  timely 
manner . 

1.     For  those  patients  known  to  be  Federal  beneficiaries  at  the 
time  of  admission,  are  decisions  to  certify  or  deny 
admission/ continued  stay  completed  within  the  program 
timeframes? 

Comment :     This  question  addresses  the  timeliness  of  making 
review  decisions  for  admission  or  continued  stay  within  the 
program  requirements,  e.g.,  within  2  working  days  of  admission 
or  by  the  expiration  of  the  initially  assigned  length-of-stay . 
An  answer  to  this  question  which  falls  out  of  the  acceptable 
range  may"  suggest  the  need  for:     a)  additional  review 
personnel  or  modification  in  review  personnel  schedules; 
b)  improvements  in  the  "tickler'1  file;  or  c)  improvements  in 
the  communication  system  between  reviewing  personnel  and/or 
the  attending  physician. 

e.  Sub-objective:     To  determine  if  the  hospital  follows  the  program 

procedures  for  denial  of  admission/continued  stay. 

1.     For  cases  denied  admission/continued  stay,  is  the  attending 
physician  given  an  opportunity  to  express  his  views  to  the 
Physician  Advisor  prior  to  the  initial  adverse  determination? 
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2.  Are  appropriate  peer  practitioners  involved  in  the  denial 
decisions? 

3.  Are  denial  notices  appropriately  distributed,  do  they  contain 
the  required  information,  and  are  they  provided  no  later 
than  2  working  days  after  admission  or  on  or  before  the 
expiration  of  the  initial  or  subsequently  assigned  length 

of  stay? 

f.     Sub-objective:     To  determine  if  the  hospital  follows  the  program 

procedures  for  handling  reconsideration  requests. 

1.     In  hospitals  delegated  this  function,  are  reconsideration 
requests  handled  according  to  program  requirements? 

go     Sub-objective:     To  determine  if  the  hospital  review  personnel 

provides  timely  information  to  the  discharge 
planning  staff. 

1.  What  percentage  of  patients  who  are  denied  admission 
certification  have  a  length-of-stay  of  6  or  more  days; 
what  percentage  of  patients  who  are  denied  continued  stay 
have  a  length-of-stay  greater  than  3  days  beyond  the  number 
of  days  certified?    Are  these  lengths-of-stay  due  to  untimely 
notification  to  the  discharge  planning  staff? 

2.  What  is  the  readmission  rate  of  patients  and  is  there  any 
relationship  between  the  rate  and  the  timeliness  of  the  dis- 
charge planning  process? 

Comment :     These  questions  seek  to  determine  the  adequacy  of 
the  link  between  concurrent  review  and  a  hospital's  dis- 
charge planning  effort.     In  order  to  provide  information  to 
the  discharge  planning  staff  and  to  monitor  the  discharge 
planning  process,  a  close  working  relationship  is  necessary 
between  the  concurrent  review  staff  and  the  hospital's 
discharge  planning  staff.     The  first  question  attempts  to 
measure  the  adequacy  of  the  link  between  concurrent  review  and 
discharge  planning  by  examining  the  length-of-stay  of  cases 
denied  admission  or  continued  stay.     The  assumption  is  that 
lengths-of-stay  beyond  the  dates  of  denial  could  have  occurred 
because  the  Review  Coordinator  did  not  initiate  discharge 
planning  early  enough  during  the  patients'  stay.     Six  days  is 
a  suggested  benchmark  in  order  to  account  for  2  working  days  to 
review  the  admission  and  issue  the  initial  adverse  determination 
and  3  working  days  for  rendering  a  reconsideration  decision. 
Three  days  is  a  suggested  benchmark  for  continued  stay  in  order 
to  account  for  the  rendering  of  a  reconsideration  decision. 
The  second  question  seeks  to  determine  the  magnitude  of  the 
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readmission  rate  of  patients  and  whether  this  is  associated 
with  problems  in  the  discharge  planning  process.  Analysis 
of  the  readmission  rate  could  serve  as  a  basis  for  subse- 
quent performance  of  a  Medical  Care  Evaluation  study.  Answers 
to  these  questions  which  fall  out  of  the  acceptable  range  may 
suggest  the  need  to:     a)  improve  the  procedures  for  notification 
to  the  discharge  planning  staff  or  b)  through  pinpointing  pro- 
blems, encourage  appropriate  corrective  actions  by  the  hospital. 
Additionally,  an  unsatisfactory  answer  to  the  first  question 
may  be  due  to  the  lack  of  alternative  beds  within  the  community  - 
a  problem  which  is  outside  the  control  of  the  hospital. 


OBJECTIVE:     TO  DETERMINE  WHETHER  REVIEW  DECISIONS  ARE  APPROPRIATE 

a.     Sub-objectives:     To  determine  if  appropriate  clinical  judgment 

is  applied  in  making  review  decisions. 

1.  What  is  the  pattern  in  the  number  of  cases  certified  or  denied? 

2.  Are  admission  or  continued  stay  certifications  justified  by 
adequate  conformance  to  PSRO  criteria  or  by  appropriate 
judgments  of  the  Physician  Advisor? 

Comment :     The  first  question  is  asked  as  a  general  inquiry  into 
the  pattern  of  cases  certified  or  denied.    No  "right"  number 
can  be  ascertained.     One  hospital  may  reach  optimum  utilization 
where  all  cases  admitted  are  medically  necessary  and 
appropriate  while  another  may  determine  that  five  percent  of  the 
admissions  are  medically  unnecessary.     By  asking  the  first 
question,  the  PSRO  is  monitoring  the  consistency  over  time  of 
the  review  decisions.     The  second  Question  is  provided  in 
order  to  validate  review  decisions  for  both  admission  and 
continued  stay  and  would  generally  be  answered  through  review 
of  medical  records. 

3.  Does  the  length-of-stay  assignment  for  various  patients  with 
a  similar  diagnosis  or  problem  reflect  the  variation  in 

the  patients'  clinical  presentation? 
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4.     For  cases  where  ill-defined  problems  or  multiple  diagnoses 
exist,  are  initial  lengths-of-stay  assigned  on  a  case  by 
case  basis? 

Comment :     It  is  expected  that  individual  differences  in 
patients  will  be  a  factor  in  review  decisions  and  satis-  ' 
factory  answers  to  questions  three  and  four  suggest  that 
there  is  an  appropriate  blend  between  the  use  of  criteria  to 
objectify  review  and  the  need  to  take  individual  patient 
variations  into  account  in  assigning  lengths-of-stay.' , 

OBJECTIVE:     TO  DETERMINE  WHETHER  REVIEW  PROCEDURES  ARE  PERFORMED 
IN  AN  EFFICIENT  MANNER 

a.     Sub-objective:     To  determine  if  the  hospital  is  making  appropriat 

use  of  review  personnel.  .  ■ 

1.     For  every  100  Federal  discharges,  what  is  the  comparison  . 
within  the  hospital  of:     a)  cases  certified  at  admission  by 
the  Review  Coordinator  and  b)  cases  referred  to  and  cer- 
tified by  the  Physician  Advisor?    How  do  these  data  compare 
with  the  data  of  other  delegated  hospitals  in  the  PSRO  area? 

Comment :     These  questions  seek  to  determine  the  pattern  of 
the  admission  certification  caseload  either  within  a  hospital 
which  has  more  than  one  Review  Coordinator  and/or  Physician 
Advisor  or  among  hospitals  with  only  one  Review  Coordinator 
and  Physician  Advisor.     Comparisons  of  level  of  certifi- 
cation, for  example,  the  percentage  of  referrals  of  several 
Review  Coordinators  within  a  hospital  may  serve  as  an 
indicator  of  the  efficiency  in  the  assigned  caseload  and 
of  the  review  process.     Excessive  variation  in  the  answers 
to  these  questions  may  suggest  the  need  to:     a)  adjust  the 
staffing  levels  for  Review  Coordinators  or  Physician  Advisors 
b)  improve  or  expand  the  screening  criteria;  or  c)  increase 
the  training  for  review  personnel.     The  problem  may  be  due 
to  the  lack  of  adequate  referral  to  the  Physician  Advisor 
of  questionable  cases  or,  conversely,  too  many  referrals 
to  the  Physician  Advisor  may  be  due  to  a  pattern  of 
insufficient  documentation  supporting  attending  physicians' 
reasons  for  admission  of  patients. 
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Sub-objective:     To  determine  the  efficiency  of  the  hospital 
in  transmitting  review  information  to  the 
paying  agency. 

1.  Has  there  been  a  significant  number  of  instances  in  which 
the  review  information  was  not  received  by  the  hospital 
billing  personnel  in  a  timely  manner? 

2,  Have  there  been  any  instances  where  the  information  sub- 
mitted on  or  as  part  of  the  bill  to  the  paying  agency 
differs  from  the  PSE.0  review  decisions? 

Comment :     These  two  questions  relate  to  the  efficient 
flow  of  information  from  the  Review  Coordinator  to  the 
hospital  billing  department  and  from  the  hospital  to  the 
paying  agency.     Question  one  is  aimed  at  determining 
whether  there  are  delays  in  forwarding  review  information 
to  the  billing  department  and,  if  so,  whether  this  has 
a  negative  impact  on  the  hospital's  ability  to  submit 
its  bills  to  the  paying  agency  in  a  timely  manner. 
Question  two  is  aimed  at  determining  whether  review  infor- 
mation submitted  by  the  hospital  is  compatible  with  the 
PSRO  review  decisions.    Answers  that  fall  out  of  the 
acceptable  range  may  suggest  the  need  to  improve  the  pro- 
cedures for  the  Review  Coordinator  to  forward  the  PSRO 
information  to  the  billing  department. 

Sub-objective:  To  determine  whether  the  patient  abstracts  are 
accurately  completed  and  efficiently  forwarded 
to  the  PSRO  and/or  data  processing  organization. 

1.    Are  patient  abstracts  accurately  completed  and  are  they 
submitted  to  the  PSRO  and/or  data  processing  organization 
within  the  agreed  upon  timeframe? 

Comment :    Additional  training  of  abstracting  personnel  may 
be  in  order  if  there  is  an  unacceptable  error  rate  in 
the  abstracts. 

Sub-objective:     To  determine  if  the  hospital  is  performing 
concurrent  review  within  an  acceptable  cost 
range . 

1.     What  is  the  comparison  by  delegated  hospitals  and  between 
delegated  and  non-delegated  hospitals  of  the  average  re- 
view cost  per  Federal  discharge? 
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Comment :     This . question  addresses  the  efficiency  of  review 
costs  and  is  provided  as  a  general  investigative  statement 
for  the  PSRO  to  monitor  costs  for  performing  concurrent 
review.     The  average  review  cost  would  include  personnel 
costs  as  identified  on  Federal  Forms  BQA  151  and  153. 

OBJECTIVE:     TO  DETERMINE  WHETHER  THE  MANAGEMENT  OF  THE  REVIEW 
SYSTEM  IS  ADEQUATE 

a.     Sub-objective.     To  determine  whether  the  peer  review  committee 

is  carrying  out  its  responsibilities  for 
managing  the  concurrent  review  system. 

1.  Does  the  committee  utilize  information  on  concurrent  review 
provided  by  the  Review  Coordinator  or  Physician  Advisor 

or  from  profiles,  if  available,  in  order  to  make  improve- 
ments in  the  review  system?    For  example,  has  the  committee 
identified  any  screening  criteria  and/or  norms  that  should 
be  added,  deleted,  or  modified? 

2.  Does  the  committee  monitor  the  performance  of  Review 
Coordinator (s)  and  Physician  Advisor (s)?     If  so,  have  there 
been  any  problems  that  required  corrective  action  by  the 
committee? 

3.  How  often  does  the  committee  hold  meetings  and  who  generally 
attends,  e.g.  Review  Coordinator,  Physician  Advisor,  other 
Physicians,  hospital  management  staff? 

A.     Are  adequate  records  maintained  to  support  the  work  of 
the  committee? 

5.  How  does  the  committee  keep  the  governing  board,  hospital 
administrator,  and  medical  staff  informed  of  issues  and 
results  of  the  concurrent  review  system? 

6.  How  does  the  committee  encourage  physician  and  other 
health  care  professionals  to  participate  in  the  PSRO 
program? 


Section  B.     Monitoring  Performance  of  Medical  Care  Evaluation  Studies 


Part  I.    Monitoring  Through  On-Site  Review  of  an  MCE  Study 

1.     OBJECTIVE:     TO  DETERMINE  WHETHER  THE  MCE  STUDIES  CONDUCTED  MAXIMIZE 
THE  POTENTIAL  FOR  IMPROVING  QUALITY  OF  CARE 

a.     Sub-objective:     To  determine  the  significance  of  the  MCE  study  topic. 

1.  Does  the  hospital  have  a  mechanism  for  prioritizing  MCE  study 
topics?     If  yes,  is  the  priority  list  used? 

Comment :     PSROs  should  encourage  hospitals  to  develop  a  mechanism 
whereby  physicians  and  nonphysician  health  care  practitioners  may 
submit  suggestions  for  planning  the  types  of  MCE  studies  that  will 
be  conducted  by  the  hospital  during  the  forthcoming  calendar  year 
or  other  time  period.     Such  a  planning  mechanism  should  result  in  a 
list  of  MCE  study  topics  for  the  hospital,  prioritized  to  reflect 
those  areas  of  health  care  practice  or  administration  which  have  the 
greatest  potential  for  yielding  study  results  which  will  significantly 
improve  the  quality  of  care  within  the  facility.     Presence  and  use  of 
such  planning  reflects  the  capability  of  the  hospital  to  organize  its 
MCE  study  effort  to  have  maximum  effectiveness. 

2.  Was  the  MCE  study  topic  generally  selected  based  on  at  least  one 
of  the  following  criteria:     i)  large  number  of  physicians  or 
patients  were  affected;  ii)  proper  medical  or  other  health  care 
practitioner  intervention  can  significantly  alter  the  health  of 
the  patients  or  improve  or  prevent  disability;  iii)  significant 
chance  for  a  severe  problem  in  the  absence  of  appropriate  medical 
or  other  health  care  practitioner  intervention;  or  iv)  generally 
agreed  upon  practices  in  handling  the  problem,  without  excessive 
controversial  aspects. 

Comment :     MCE  study  topics  which  meet  at  least  one  of  these  criteria 
are  most  likely  to  result  in  significant  impact  on  the  quality  of 
health  care  in  the  hospital. 

3.  If  aggregated  data  were  used  to  select  the  topic,  e.g.,  concurrent 
review  data,  profile  analysis,  hospital  data,  did  the  data 
sufficiently  document  the  significance  of  the  problem  area  selected 
for  study? 

Comment :     Such  data  can  often  indicate  areas  of  health  care  where 
potential  problems  exist  and  where  indepth  analysis  through  an  MCE 
study  will  have  substantial  impact  on  improvement  of  care.  Wherever 
available,  such  data  should  be  used  by  the  hospital  to  help  select 
and  prioritize  useful  MCE  study  topics. 
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4.  Was  the  objective(s)  of  the  study  clearly  defined? 

Comment :     The  MCE  study  topic  should  be  directed  at  specific  goals 
and  objective (s) ,  preferably  clearly  stated  at  the  outset.  The 
objective  should  be  stated  in  "action"  oriented  terms  (i.e.  to 
improve,  to  evaluate  or  assess,  to  reduce,  to  assure,  etc.)  in  order 
to  emphasize  the  commitment  to  change,  where  necessary,  to  maintain 
or  improve  the  quality  of  health  care. 

5.  Was  the  study  topic  defined  in  a  way  which  allowed  identification 
of  a  homogeneous  study  population? 

Comment :     A  well-defined,  homogeneous  study  population  is  essential 
to  ensure  that  the  patients  selected  reflect  the  objectives  of  the 
study.     In  general,  broad  topics  such  as  "diabetes"  or  "stroke" 
do  not  lend  themselves  to  clear  objectives  nor  address  study 
populations  which  are  homogeneous  to  allow  effective  identification 
of  problems  in  health  care. 

Several  examples  of  study  topics  which  include  the  objective  and  allow 
identification  of  homogeneous  population  are: 

1.  Diabetes  Mellitus :     To  assess  the  diagnosis  and  management  of  newly 
diagnosed  juvenile  diabetics. 

2.  Appendicitis-Perforated :     To  reduce  the  time  interval  between 
hospital  admission  and  surgical  intervention. 

3.  Myocardial  Infarction:     To  assure  prompt  and  accurate  diagnosis 
and  treatment  of  myocardial  infarction  in  the  first  72  hours  of 
hospitalization  for  those  patients  admitted  through  the 
emergency  room  with  a  suspected  diagnosis  of  Acute  Myocardial 
Infarction. 

Sub-objective:     To  determine  the  appropriateness  of  the  criteria  and 
standards . 

1.     Do  the  criteria  relate  directly  to  the  objective  of  the  study? 

Comment :     Each  criterion  utilized  in  the  study  must  be  specifically 
related  to  the  objective  of  the  study  and  should  be  useful  in  discrimi- 
nating instances  of  clearly  optimal  care  from  those  of  clearly 
unacceptable  care.     For  example,  the  criterion  "abdominal  X-ray  ordered" 
has  little  relevance  to  a  study  with  the  goal  of  assessing  care  rendered 
to  patients  hospitalized  in  diabetic  coma.     Similarly,  in  a  study  of 
a  group  of  patients  with  pulmonary  edema,  the  criterion  "intravenous 
five  percent  dextrose  in  water  started"  will  allow  identification  of 
cases  where  normal  saline  may  have  been  administered ,  whereas  the 
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criterion  "intravenous  started"  will  not  make  this  distinction. 

2.    Was  each  criterion  specific  to  one  variable?    Were  the  criteria 
stated  in  explicit,  measurable  terms?    Were  they  stated  in  terms 
understandable  to  an  abstractor? 

Comment :  The  PSRO  should  review  each  of  the  study  criterion  to  see 
if  the  following  are  present: 

i.     Each  criterion  should  address  a  single  element  or  variable: 


Poor  Criteria 


Good  Criteria 


1.     Diastolic  pressure  over 
100  and  hypertension 
previously  untreated 


Diastolic  pressure  over 
100  on  2  consecutive 
readings 


2.     Hypertension  previously 
untreated 

ii.     Each  criterion  should  be  stated  in  measurable  rather  than 
descriptive  terms: 


Poor  Criteria 


Good  Criteria 


1.     Routine  vital  signs 
post-op 


1.     Blood  pressure,  respirations 
and  temperature  recorded  every 
4  hours  during  first  24  hours 
following  surgery 


2.     No  acetone  in  urine 


2.     Urine  acetone  negative  for 
24  hours  before  discharge 


3.     Patient  not  anemic 


4.     Patient  afebrile 


3.  Hematocrit  not  less  than 

33  percent  prior  to  surgery 

4.  Temperature  less  than  99.9  F  ora 
for  24  hours  prior  to  discharge 


Elevated  WBC 


WBC  over  10,000 


Each  criterion  should  be  stated  explicitly  to  eliminate  any 
possibility  of  misinterpretation  on  the  part  of  nonphysician  data 
abstractors.     For  example,  the  criterion  "post-operative  chest  X-ray 
should  be  revised  to  read  "no  mention  of  atelectasis  on  post-opera- 
tive chest  X-ray  report." 
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3.     Were  the  criteria  and  standards  subject  to  comment  and/or  approval 
by  a  group  representative  of  those  professionals  whose  care  was 
subject  to  review? 

Comment :     In  most  cases,  an  initial  set  of  draft  criteria  and 
standards  will  be  developed  by  a  few  members  of  the  hospital  staff 
or  Audit  or  MCE  Study  Committee.     However,  the  criteria  should  then 
be  distributed  for  comment  to  those  whose  practice  will  be  audited. 
This  is  a  key  step  in  the  MCE  study  process  and  has  several  important 
benefits:     i)  aids  in  clarifying  and  developing  the  best  set  of 
criteria;  ii)  maximizes  commitment  to  study  findings  and  recommendations 
iii)  serves  as  an  important  tool  for  education  and  information  exchange 
which  may,  in  and  of  itself,  create  the  potential  for  improvement  in 
health  care  practice. 

In  order  to  answer  . this  question,  the  PSRO  should  determine:     1)  who 
developed  the  draft  criteria/standards  and  ii)  the  method  used  to 
distribute  the  draft  criteria/standards  for  comment. 

Sub-objective:     To  determine  the  appropriateness  of  the  study  design 

1.  Did  the  cases  selected  for  the  MCE  study  match  the  pre-defined 
patient  characteristics? 

Comment :     In  order  to  answer  this  question,  the  PSRO  should  review 
several  patient  records  to  verify  that  the  cases  selected  match  the 
pre-defined  patient  characteristics. 

2.  Was  the  sample  size  adequate  to  make  determinations  about  the 
quality  of  care?    Was  the  sample  selected  from  all  patients  and  not 
just  Title  18,  19  and  5  patients? 

Comment :    While  it  is  difficult  to  evaluate  whether  the  study  sample 
was  large  enough,  the  PSRO  using  the  ICDA  code  could  make  some  general 
comparisons  between  the  number  of  patients  during  the  given  audit 
period  which  had  the  respective  diagnosis  or  procedure  and  the  number 
of  charts  reviewed  in  the  study.     Again  this  is  a  difficult  area  to 
measure  and  conclusions  by  the  PSRO  will  more  than  likely  be  based  on 
subjective  judgment. 

For  part  b  of  this  question,  the  PSRO  should  review  a  sample  of  the 
study  charts  to  see  if  the  patients  were  limited  to  one  or  all  of  the 
three  Federal  health  programs. 

3.  Did  the  retrospective  timeframe  cover  an  adequate  period  of 

time  in  order  to  assure  selection  of  an  appropriate  number  of  cases? 

Comment :     The  timeframe  used  in  the  MCE  will  depend  on  several 
factors  including  the  topic  under  study,  admission  rate,  bed  size  of 
the  facility,  etc.     A  6  month  time  period  is  often  used  although 
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this  is  variable.     The  PSRO  should  determine  whether  the  time  frame 
was  adequate  to  yield  a  sample  large  enough  to  allow  valid  study 
results . 

4.  Was  one  individual  assigned  responsibility  for  overall  manage- 
ment of  the  MCE  study  process? 

Comment :     This  question  seeks  to  address  how  the  MCE  study  process 
was  managed.    While  one  or  more  individuals  may  jointly  coordinate 
the  study,  the  PSRO  should  determine  the  existence  of  an  organized 
management  framework  for  conduct  of  the  study  which  includes  designa- 
tion of  responsibility (ies)  to  qualified  individuals. 

5.  Were  the  appropriate  peer  physicians  and  peers  of  other  health 
care  practitioners  whose  care  was  assessed  involved  in  all  phases 
of  the  study? 

Comment:    Many  MCE  studies  provide  an  opportunity  for  multi- 
disciplinary  review  of  care.     Therefore,  the  study  committee 
should  assure  that  peer  practitioners  representing  each  of  the 
disciplines  whose  care  is  being  reviewed,  physician  and  nonphysician , 
participate  in  developing  the  criteria  and  standards  whereby  their 
care  will  be  assessed,  in  making  decisions  regarding  the  quality  of 
care  rendered  and  in  making  recommendations  for  corrective  action. 


Sub-objective:     To  determine  the  appropriateness  of  the  data  analysis 

1.  Was  the  data  displayed  in  such  a  way  as  to  help  the  Committee 
conduct  its  analysis? 

2.  Did  the  data  identify  each  case  where  variance  from  criteria  and 
standards  was  found? 

3.  Was  a  judgment  made  as  to  the  reasons  for  variance  from  criteria 
or  standards  in  each  case  where  variance  was  found? 

4.  Was  the  data  sufficient  to  support  the  conclusions  of  the  study, 
including  reasons  for  variance  from  criteria  and  standards? 

Comment :     This  set  of  questions  is  designed  to  assist  the  PSRO  in 
reviewing  the  procedures  and  evaluating  the  validity  of  the  data 
analysis  performed  to  identify  problem  areas. 

Sub-objective:     To  determine  the  appropriateness  of  recommended  actions 


1.     Were  the  recommended  actions  consistent  with  the  deficiencies 
identified  by  analysis  of  the  study  data? 
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Comment :     After  the  committee  identifies  any  inadequacies  in  care  which 
the  study  has  revealed,   it  should  prepare  recommendations  for  actions 
to  correct  the  deficiencies.     Based  on  its  review  of  the  Committee's 
data  analysis,  the  PSRO  should  review  the  recommended  actions  to 
determine  whether  they  specifically  address  the  deficiencies  which 
were  identified. 

2,  Were  the  data  analysis  and  results  of  the  study  provided  to 
appropriate  members  of  hospital  medical  staff  or  other  health 
care  practitioner  staff,  chief  administrative  officer  of  the 
hospital  and  the  chairman  of  the  board  of  trustees? 

Comment :     A  summary  of  the  study  findings,  analysis  and  recommendations 
should  be  prepared  and  distributed  to  the  above  parties  in  order  to 
begin  the  process  of  implementing  corrective  actions.     The  PSRO  should 
verify  that  the  summary  was  sent  to  the  respective  parties. 

3.  Was  an  action  plan  developed  in  order  to  implement  the  corrective 
actions? 

Comment :     An  action  plan  should  be  developed  in  order  to  implement  the 
corrective  actions.     Such  a  plan  should  include  assignment  of  responsi- 
bilities to  individuals  to  effect  the  change  and  identification  of 
the  procedures  to  monitor  implementation  of  the  corrective  actions. 
The  PSRO  should  determine  whether  such  a  plan  has  been  developed  and 
whether  the  appropriate  individuals  have  accomplished  their  responsi- 
bilities . 

Sub-objective:     To  determine  the  appropriateness  of  the  re-study 

1.  Did  the  re-study  focus  on  key  indicators  of  performance,  e.g., 
criteria  reflecting  deficiencies  in  the  original  study,  in  order 
to  identify  whether  the  recommended  actions  have  corrected  the 
deficiencies? 

2.  How  successful  was  the  hospital  in  correcting  the  deficiencies 
identified  in  the  original  study? 

Comment :     The  Committee  should  schedule  a  short  follow-up  study  sometime 
after  the  action  plan  has  been  implemented  in  order  to  determine 
whether  the  corrective  actions  improved  the  deficiencies  cited  in  the 
original  study.     This  followup  should  focus  on  those  criteria  where 
deficiencies  existed  in  the  original  study. 
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Part  II.     Monitoring  Through  Review  of  PMIS  -  MCE  Reporting  Forms 

1.     OBJECTIVE:     TO  DETERMINE  IF  PROGRAM  REQUIREMENTS  ARE  BEING  MET. 

a.     Sub-objective:     To  determine  if  the  hospital  is  submitting 

timely  and  complete  MCE  study  reports  to  the 
PSRO 

Source 

1.     Are  quarterly  BQA  forms  131,  133,  and  135  1.     BQA  131,  133 

submitted  within  the  defined  timeframe  in  135 
order  to  allow  the  PSRO  to  submit  them  to 
DHEW  no  later  than  45  days  following  the 
end  of  the  quarter?    Do  they  contain  all 
the  required  information?    Are  the  forms 
clear  and  legible? 

b.  Sub-objective:     To  determine  the  hospital's  compliance  with  the 

required  number  of  MCE  studies 

1.  What  is  the  pattern  in  the  hospital's  1.     BQA  135 
productivity  in  terms  of  numbers  and 

frequency  of  MCE  studies?    Are  studies 
being  conducted  on  a  regular  basis?  Do 
the  number  of  studies  being  conducted 
meet  DHEW  requirements? 

2.  Are  re-studies  performed  for  those  MCE  2.     BQA  135 
studies  which  formulated  recommended 

actions  to  correct  deficiencies? 

c.  Sub-objective:     To  determine  whether  the  hospital  is  conducting 

MCE  studies  and  re-studies  within  a  reasonable 
period  of  time 

1.  Does  the  hospital  complete  all  phases  of  1.     BQA  135 
each  MCE  study  through  formulation  of 

recommended  actions  within  a  6  month 
interval? 

2.  Is  the  hospital  completing  at  least  a  2.     BQA  135 
majority  of  (or  nearly  all)  MCE  studies 

within  the  original  estimated  timeframes? 
If  not,  are  the  reasons  justifiable? 

3.  Where  multiple  hospitals  study  the  same  3.     BQA  135 
subject,  are  there  unexplainable 

differences  in  the  timeframe  variations? 

4.  Do  re-studies  take  place  within  a  reasonable      4.     BQA  135 
time  but  usually  no  later  than  1  year 

following  performance  of  the  initial  study? 


ft 
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5.     Are  nearly  all  re-studies  completed  within 
the  original  estimated  timeframe? 

d.     Sub-objective:     To  determine  if  the  hospital  is  appropriately 

reporting  MCE  study  personnel  costs 

1.     On  an  annual  basis,  are  the  person  hours  identified  for  MCEs 
consistent  with  the  expenditure  of  MCE  study  personnel  funds? 

Sources:     BQA  153  -  Quarterly  Delegated  Hospital  Function 
Cost  Summary 

BQA  131,  133  -  Item  No.  20  and  No.  5,  respectively, 
Person  Hours  Utilized 

Method  (example) : 

Based  on  eight  MCE  studies 

BQA  153  on  an  annual  basis  indicates:     $  3,000  -  physician  costs 

5,000  -  other  per.  costs 
$  8,000 

Annual  totals  on  BQA  131,  133  indicate:       48  physician  hours 

120  other  person  hours 

c.    Multiply  the  hours  x  the  hourly  rate  identified  by  the 
hospital  for  physicians  and  other  persons: 

48  hours  x  $35.00  per  hour      +      $  1,680 
120  hours  x  $15.00  per  hour      =      $  1,800 

$  3,480 


a. 
b. 


d.     Significant  differences  between  the  costs  identified  on 
BQA  153  and  through  the  above  analysis  should  lead  to 
further  followup  by  the  PSRO. 


• 


